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Abstract 

Background: The present study examines the structure and operation of social networks of information and 
advice and their role in making decisions as to whether to adopt new evidence-based practices (EBPs) among 
agency directors and other program professionals in 12 California counties participating in a large randomized 
controlled trial. 

Methods: Interviews were conducted with 38 directors, assistant directors, and program managers of county 
probation, mental health, and child welfare departments. Grounded-theory analytic methods were used to identify 
themes related to EBP adoption and network influences. A web-based survey collected additional quantitative 
information on members of information and advice networks of study participants. A mixed-methods approach to 
data analysis was used to create a sociometric data set (n = 176) for examination of associations between advice 
seeking and network structure. 

Results: Systems leaders develop and maintain networks of information and advice based on roles, responsibility, 
geography, and friendship ties. Networks expose leaders to information about EBPs and opportunities to adopt 
EBPs; they also influence decisions to adopt EBPs. Individuals in counties at the same stage of implementation 
accounted for 83% of all network ties. Networks in counties that decided not to implement a specific EBP had no 
extra-county ties. Implementation of EBPs at the two-year follow-up was associated with the size of county, urban 
versus rural counties, and in-degree centrality. Collaboration was viewed as critical to implementing EBPs, especially 
in small, rural counties where agencies have limited resources on their own. 

Conclusions: Successful implementation of EBPs requires consideration and utilization of existing social networks 
of high-status systems leaders that often cut across service organizations and their geographic jurisdictions. 

Trial Registration: NCT00880126 



Background 

Each year, about 6% of U.S. children and adolescents 
receive some form of mental health care at an annual 
cost of more than $11 billion [1]. Despite the increased 
availability and demand for evidence-based practices 
(EBPs) for the treatment of youth mental health and 
behavioral problems [2-5], 90% of publicly funded child 
welfare, mental health, and juvenile justice systems do 
not use EBPs [6]. The reasons for this lack of use and the 
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characteristics of systems that predict successful imple- 
mentation of EBPs remain poorly understood. 

Interpersonal contacts within and between organizations 
and communities are important influences on the adop- 
tion of new behaviors [7-9]. Based on Diffusion of Innova- 
tions Theory [7] and Social Learning Theory [10], 
Valente's [11] social-network thresholds model calls for 
the identification and matching of champions within peer 
networks that manage organizational agenda setting, 
change, and evaluation of change (e.g., data collection, eva- 
luation, and feedback). Studies and meta-analyses have 
also shown that both the influence of trusted others in 
one's personal network and having access and exposure to 
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external information are important influences on rates of 
adoption of innovative practices [12-16]. 

Sociometric techniques for capturing the structure of 
such networks have been used to study patterns of diffu- 
sion of innovations in several arenas, including tobacco 
prevention programs, contraceptive use and family plan- 
ning, HIV prevention, and clinical practice guidelines 
[17]. However, to our knowledge, these techniques have 
not been used to study the implementation of EBPs in 
child welfare and child mental health. In addition, these 
techniques are limited in providing depth of understand- 
ing to the process of implementation and to the context 
in which these influence networks operate. Such depth 
is usually provided through the application of qualitative 
methods [18]. 

Using both quantitative and qualitative data, we sought 
to accomplish the following: (1) describe the structure 
and operation of information and advice networks of 
public youth-serving systems in 12 California counties 
and (2) determine the influence of these networks in the 
implementation of an evidence-based intervention 
designed to reduce placement in group and residential 
care, juvenile arrest rates, substance abuse, youth vio- 
lence, and child behavioral and mental health problems. 

Methods 

Setting 

The present study uses data from the Cal-40 Study, a 
clinical trial of an implementation strategy to scale up 
the use of an EBP for treatment of externalizing beha- 
viors and mental health problems [19,20]. This EBP, 
called Multidimensional Treatment Foster Care (MTFC) 
[21], has been shown to reduce out-of-home placement 
in group and residential care, juvenile arrests, substance 
abuse, youth violence, pregnancy, and behavioral and 
emotional problems. The implementation method being 
tested is the use of community-development teams 
(CDTs) [22] to scale up MTFC in public youth-serving 
systems in California; control sites obtain technical assis- 
tance for implementing MTFC without the use of CDTs. 
The Cal-40 study targeted 40 California counties that 
had not already adopted MTFC. They were matched to 
form three nearly equivalent groups. The matched 
groups were then randomly assigned to three sequential 
cohorts in a wait-list design with staggered start-up time- 
lines (at months 6, 18, or 30). Within each cohort, coun- 
ties were randomly assigned to CDT or standard 
implementation conditions, thereby generating six repli- 
cate groups of counties, with three assigned to CDT. 
Across 40 counties, participants are approximately 600 
system leaders, agency directors, and practitioners; 400 
foster parents; and 900 youth and their families. 

Progress toward implementation was assessed by 
means of a stage-of-implementation checklist (SIC) [19]. 



Multiple indicators are used to measure both progres- 
sion through the stage and quality of participation of 
the individuals involved at each stage. Stages 1-3 track 
the site's decision to adopt/not adopt MTFC, the feasi- 
bility of adoption, their readiness, and the adequacy of 
their planning to implement. In stage 4, recruitment and 
training of the MTFC treatment staff (i.e., program 
supervisor, family therapist, individual therapist, foster 
parent trainer/recruiter, and behavioral skills trainer) 
and foster parents are measured. Stage 5 tracks the 
training and implementation of procedures to measure 
fidelity of MTFC use. Stage 6 tracks services and consul- 
tation to services, including dates of first placement, 
consult call, clinical meeting, and foster parent meeting. 
Stage 7 tracks ongoing services, consultation, and fidelity 
monitoring and how sites use those data to improve 
adherence. Stage 8 evaluates the site's competency in 
the domains required for certification as an independent 
MTFC program. 

Design 

We used a mixed-method design that is both exploratory 
(i.e., by developing the conceptual model of systems leader 
information and advice networks) and confirmatory (i.e., 
by testing the conceptual model) [23], achieving three 
types of integration of quantitative and qualitative data: (1) 
convergence: using both types of data to answer the same 
question; (2) complementarity: using each type of data to 
answer related questions, where the type of data is specific 
to the question asked (e.g., using qualitative data to gener- 
ate hypotheses, provide depth of understanding, and focus 
on the function and context of social networks and quanti- 
tative data to confirm hypotheses, provide breadth of 
understanding, and focus on social-network structure and 
predictors of implementation stage); and (3) expansion: 
using one type of data to address questions raised by the 
use of the other type of data (e.g., using qualitative data to 
explain results of quantitative analyses) [18]. 

Study sample 

Participants for the current study included members of 
the influence networks of the agencies that comprised 
the first cohort of counties (n = 13) of the Cal-40 Study. 
As of October 2010, two counties had declined to partici- 
pate in the study; two counties had reached each of SIC 
stage 1 (engagement), stage 2 (consideration of feasibil- 
ity), stage 3 (readiness planning), stage 6 (services and 
consultation to begin), and stage 7 (ongoing services, 
model fidelity and feedback); and one county had reached 
stage 8 (competency/certification/licensure). A purpose- 
ful sampling strategy was employed, beginning with 
directors of the child welfare, mental health, and proba- 
tion departments of all 13 counties. In some instances, 
associate directors or senior program managers were 
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recommended by the directors to be interviewed in their 
place. 

Of the 45 administrators from all 13 counties invited to 
participate, 38 representing 12 counties agreed to do so, 
yielding a response rate of 84%. Each participant com- 
pleted a semistructured interview conducted between July 
and September 2008, with the number of interviews per 
county ranging from two to six. Twenty-eight participants 
were interviewed face-to-face; 10 were interviewed by tele- 
phone. All those interviewed were then asked to complete 
a web-based survey to identify individuals on whom they 
relied for advice regarding EBP implementation. Thirty 
individuals (86%) of those who participated in semistruc- 
tured interviews also completed the web-based survey. 
Data on network ties from the web-based survey were 
supplemented by additional data provided in participants' 
qualitative interviews. After complete description of the 
study to the participants, written informed consent was 
obtained. The research study was approved by the Institu- 
tional Review Board at the University of Southern 
California. 

Data collection 

The semistructured interview centered on knowledge and 
implementation of MTFC and other EBPs at the county 
level. Interviewees were asked if they had ever heard of 
the Cal-40 Study or MTFC and what their motivations 
were to participate or not participate in the program. 
Participants were then asked who they had talked to 
about participation in MTFC or other EBPs; prompts 
were given to participants as necessary to identify who 
they talked to, their relationship to that person, their rea- 
sons for talking to that person, and the amount of influ- 
ence that person had on their decision to participate in 
implementing MTFC or a similar EBP. Then participants 
were asked about collaborations both within and between 
county agencies (child welfare, mental health, probation) 
and the nature of these collaborations. Specifically, parti- 
cipants were asked to identify what made for a successful 
versus an unsuccessful collaboration. Finally, participants 
were asked about who usually suggested that their agency 
take on new programs or initiatives. Probes for influential 
network actors included agency staff, other agencies, 
community-based organizations, other county officials, 
etc. 

The web-based survey asked participants to provide 
general demographic information (i.e., gender, age, num- 
ber of years in occupation, current position, and time 
with agency). Per criteria established by Valente and col- 
leagues [15,24], each study participant was asked to 
identify up to 10 individuals on whom they relied for 
advice about whether and how to use EBPs for meeting 
the mental health needs of youth served by their agency. 



Data analysis 

A methodology of "Coding Consensus, Co-occurrence, 
and Comparison" outlined by Willms and colleagues [25] 
and rooted in grounded theory (i.e., theory derived from 
data and then illustrated by characteristic examples of 
data) [26] was used to analyze semistructured interviews. 
Audio-recorded interviews were transcribed, and lists of 
codes were developed by each investigator and then 
matched and integrated into a single codebook. Each text 
was independently coded by at least two investigators and 
disagreements in assignment or description of codes was 
resolved through discussion between investigators and 
enhanced definition of codes. The final list of codes or 
codebook, constructed through a consensus of team mem- 
bers, consisted of a numbered list of themes, issues, 
accounts of behaviors, and opinions that related to organi- 
zational and system characteristics that influence imple- 
mentation of MTFC. The transcripts were then assessed 
for agreement between the authors on the coding, based 
on a procedure used in other qualitative studies [27]. 
Inter-rater reliability was assessed for a subset of pages 
from 10 transcripts. For all coded text statements, the 
coders agreed on the codes 91% of the time (range = 88%- 
94%), indicating good reliability in qualitative research 
[27]. The computer program NVivo (QSR International, 
Cambridge, MA, USA) [28] was used for coding and then 
to generate a series of categories arranged in a treelike 
structure connecting text segments grouped into separate 
categories of codes, or "nodes." These nodes and trees 
were used to further the process of axial and pattern cod- 
ing to examine the association between different a priori 
and emergent categories. 

The matrix of ties used to analyze advice networks was 
constructed from data collected from the web-based sur- 
vey, supplemented by data collected during the qualitative 
interviews. The social-network analysis proceeded in three 
stages: network visualization, structural analysis, and sta- 
tistical analysis of outcomes. The network visualization 
was accomplished using NetDraw 2.090 (Analytic Tech- 
nologies, Lexington, KY, USA). The spring embedder rou- 
tine was used to generate the network visualizations 
presented in Figure 1 [29] . 

Structural analyses were then conducted on these net- 
work data using Ucinet for Windows, Version 6 (Analytic 
Technologies, Harvard, MA, USA) [30] . Several network- 
level measures of structure were assessed, including total 
number of ties, network size, density (the number of 
reported links divided by the maximum number of possi- 
ble links), average distance between nodes, and the num- 
ber of components (i.e., unique subnetworks). While there 
are a host of possible metrics from which to choose, we 
opted for a set of common network metrics in order to 
provide a descriptive presentation of the network, based 



Palinkas ef al. Implementation Science 201 1, 6:1 13 
http://www.implementationscience.eom/content/6/1/1 13 



Page 4 of 1 1 




Figure 1 Evidence-based practice advice networks by implementation stage Advice Network Properties. Grey nodes represent individuals 
who reported on the stage-of-implementation checklist as being in stages 0-1, blue-green nodes represent those in stages 2-6, and bright green 
nodes those in stages 7-8. White nodes depict individuals about whom insufficient information was obtained to ascertain implementation stage 
or about whom implementation stage is not relevant, such as individuals who work for the California Institute of Mental Health. 
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on our analysis of data collected from the semistructured 
interviews. To assess status and interconnectivity within 
the network, we calculated degree centrality for both 
incoming ties (being nominated by alters) and outgoing 
ties (nominating alters). In-degree and out-degree central- 
ity scores assess the relative status of a given node. We 
also examined several other measures of network status, 
including between-ness, closeness, and eigenvector cen- 
trality. With the exception of eigenvector centrality, these 
measures were not associated with implementation and 
were dropped from further analyses. Eigenvector centrality 
also allows one to examine in-ties relative to out-ties, but 
in- and out-degree centrality correspond directly to counts 
of nominations by and toward an actor and, as such, have 
a straight-forward substantive interpretation, which eigen- 
vectors lack. In-degree and out-degree centrality need not 
be correlated and, in this network, are not. In-degree cap- 
tures the status of a node in a network by assessing how 
frequently that node is nominated by others in the net- 
work. This measure reflects how important others in the 
network perceive a given node to be. Out-degree assesses 
the involvement of a node in a network by measuring how 
many others a given node nominates, which may have lit- 
tle to do with how others in that network perceive that 
node. 

Homophily (i.e., likeness between individuals in a net- 
work based on specified criteria) data were assessed on 
three key variables of interest identified during the semi- 
structured interviews: county, agency, and MTFC imple- 
mentation stage. Homophily scores were created using 
an algorithm that divided the total number of like ties for 
that individual based on each of the criteria above by the 
total ties in that individual's network. Scores ranged 
between 0 (no homophily) and 1 (perfect homophily). 
This score can be regarded as the proportion of indivi- 
duals in a person's network who share a characteristic 
(i.e., county, agency, implementation stage of MTFC) 
with that individual. We selected these metrics to assess 
homophily because our analysis of the qualitative data 
from the interviews led us to hypothesize that persons in 
relative proximity to one another (i.e., same agency or 
same county) would be more apt to communicate. More- 
over, we hypothesized that organizations at similar levels 
of MTFC implementation would be in contact with one 
another, in part due to their shared stage of adoption. 

We used ordinary least-squares multivariate regression 
models to assess stage of implementation achieved at the 
two-year follow-up (October 2010) as a function of net- 
work- and individual-level properties. Centrality scores 
calculated in Ucinet were merged with the original data 
set. We then regressed implementation stage on in-degree 
and out-degree centrality, adjusting for two county-level 
dummy variables representing large versus small size and 
urban versus rural. These analyses were designed to 



understand how implementation stage varied as a result of 
position within the sociometric network. Social-network 
data are derived from nonindependent observations and 
present a challenge to statistical analysis. To deal with this 
issue, we employed the most common approach, which is 
to use a program such as UCINET to generate position- 
specific variables, which subsequently can be exported to 
the original individual-level database and analyzed with 
standard linear models [e.g., [31]]. In cases where the out- 
comes occur at the level of the tie (not at the level of the 
node as in the present context), hierarchical linear models 
with random effects can be employed, which model node- 
level and tie-level properties as two levels of analysis [e.g., 
[32]]. As autocorrelation was not found in our data, the 
issue of independence is primarily a conceptual one. 

Results 

Characteristics of study participants are described in 
Table 1 below. Participants in the study were middle- 
aged (mean age = 49.38 years), and nearly two-thirds 
were female (60.5%). Type of agency was evenly divided 
between child welfare, mental health, and probation. 
Fourteen of those interviewed were agency directors, 8 
were assistant directors, and 16 were program managers. 
These participants hailed from both large and small 
counties that were urban and rural and were located 
throughout the state of California. 

Structure and function of influence networks 

Analysis of interview transcripts revealed that systems 
leaders develop and maintain networks of information 
and advice according to position in agency (e.g., directors, 
program managers), responsibility (probation, mental 
health, child welfare), geography (within a county, neigh- 
boring counties), and friendship ties (co-workers, class- 
mates). These networks expose leaders to information 
about EBPs and opportunities to adopt EBPs and influ- 
ence decisions to adopt EBPs. This information comes 
from others within the same county, including supervi- 
sors or employees within the same agency, counterparts 
in other agencies, community-based providers, and com- 
munity advocates. 

Noting both in-county and out-of-county resources for 
discussing EBPs was common across interviews. Within 
counties, participants said they drew on advice from indi- 
viduals in their own agency (although this was not sup- 
ported to a high degree by network analyses), outside 
agencies, community-based organizations, and community 
advocacy organizations. Network members located outside 
the county included professional organizations like the 
California Parole Officers Association, the Child Welfare 
Directors Association, and the California Mental Health 
Directors Association; intermediaries like the California 
Institute of Mental Health (CIMH); nonprofit foundations 
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Table 1 Participant characteristics for social-network data 
(n = 38) 

Individual characteristics 



Mean age in years (range)* 
Gender 

Male 

Female 
Agency 

Child Welfare 

Mental Health 

Probation 
Position 

Director 

Assistant Director 

Program Manager 
County characteristics 
County size 

Small 

Large 
Region 

Northern 

Bay Area 

Central 

Southern 
Rural county 

Yes 

No 

Network characteristics 

Proportion same county 

Proportion same agency 

Proportion same implementation stage 



49.36 (31 - 63) 

1 5 (39.5%) 
23 (60.5%) 

14 (36.8%) 
12 (31.6%) 
12 (31.6%) 

14 (36.8%) 
8 (21.1%) 

16 (42.1%) 



20 (52.6%) 
18 (47.4%) 

8 (21.1%) 
18 (47.4%) 
10 (26.3%) 
2 (5.3%) 

1 5 (39.5%) 
23 (60.5%) 

0.810 (0.226) 
0.381 (0.266) 
0.830 (0.223) 



Information on age was missing for eight participants. 

like the Annie E. Casey Foundation and Casey Family Pro- 
grams; universities; and consultants. Peers from other 
counties were also an important source of information 
and advice; however, this occurred more in small rural 
counties than in large urban counties. 

Among the forums for the exchange of information 
and advice about EBPs are regularly scheduled meetings 
within the county, region, and state; initiatives that 
involve contact of systems leaders by CIMH; agency 
staff; and other county agencies and community-based 
organizations. One director specifically cited a monthly 
statewide gathering as a particularly useful venue for 
gathering information on EBPs: 

"I go monthly to the Children's System of Care meeting in 
Sacramento. And that's where other people in similar 
administrative positions to myself who are responsible for 
children's mental health services, we chew on these kinds of 
things. We discuss these kinds of things. And, you know, we 
have presentations, and so forth. So that is my peer group. 
And that, um, certainly provides a lot of information to me 
in making decisions. "-Mental Health Department Director 



Systems leaders also obtain information and advice on 
EBPs from counterparts in counties widely regarded for 
serving as "models" for innovation and EBP implementa- 
tion, as one agency director noted when asked who she 
looks to outside her own county: 

"There's a. ..always [our practice of] checking with 
Orange County, LA, [when considering adopting a new 
program]. Although quite big they do some very progres- 
sive things as well. Um, and so you know which counties 
are kind of doing some leading edge, and, not just lead- 
ing edge, but that also have uh, the evaluation compo- 
nent of it. "-Chief Probation Officer 

Participants described a wide range of advice seeking in 
qualitative interviews, which included both whether to 
implement an EBP (MTFC in particular) or a new, inno- 
vative program in their county and how to best imple- 
ment such a program. Social-network survey-based ties 
between respondents included both types of advice seek- 
ing. While some participants in the qualitative interviews 
simply provided a name of someone who they had con- 
tacted about an EBP (or other program), others provided 
a more elaborate description of the advice-seeking inter- 
action. For example, several participants discussed advice 
seeking in relation to the cost and feasibility of imple- 
menting a particular program; this included instances of 
where they had decided not to implement a specific pro- 
gram because they had been informed by their counter- 
parts in other counties or directors of community-based 
organizations within their own county that the cost of 
implementation would be prohibitive. Others discussed 
advice seeking related to approaching appropriate com- 
munity partners for collaboration. 

Representations of the influence networks for exchan- 
ging information related to EBPs in general are found in 
Figure 1. Grey nodes represent individuals who reported 
being in implementation stages 0-1, blue-green nodes 
represent stages 2-6, and bright green nodes represent 
stages 7-8. White nodes depict individuals about whom 
insufficient information was obtained to ascertain imple- 
mentation stage or about whom implementation stage is 
not relevant, such as individuals who work for CIMH or 
other non-county-affiliated organizations. A simple visual 
inspection of the network diagram reveals that many of 
the nodes in this network are connected to others in 
similar implementation stages. 

Table 2 provides metrics that help to describe this net- 
work. A total of 176 individuals with 233 ties comprised 
this network. Network density was relatively low; less than 
one percent of all possible ties among nodes were present. 
We caution against over-interpreting this metric because 
mathematically, as network size increases, density 
decreases [29] . Several other metrics suggested evidence of 
connectivity. There were eight unique components, that is, 
"disconnected" sub-networks. One of these components 
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Table 2 Network metrics for combined interview and 
survey network (n = 176) 



Metric Total network 



Size 


176 


Number of ties 


223 


Density 


0.0072 


Average distance 


1.884 


Number of components 


8 


In-degree centrality 


1.27 (0.91) 


Out-degree centrality 


1 .27 (3.05) 



contained 81% of the overall network, while the remaining 
seven components ranged in size from one to nine indivi- 
duals. Individuals from 10 of the 12 counties were repre- 
sented in the largest component, and three counties were 
each represented in two or more components. Moreover, 
the average number of ties separating any two individuals 
in the network was 1.9. 

The principle of homophily was well supported for 
both county and implementation stage among members 
of the original sample. On average, 81% of network ties 
were among individuals who came from the same county, 
and 83% of network ties were among individuals who 
were classified in the same implementation stage as the 
respondent. Interestingly, only 38% of network ties were 
among individuals who came from the same county 
agency as the respondent. Taken together, these results 
indicate that individuals often rely on others from within 
their own county for advice on EBPs, although not neces- 
sarily individuals from within their agency, and from 
individuals outside their county. This latter observation 
was supported by the fact that seven counties had links 
to one individual who works for the CIMH and is known 
throughout the state as someone on whom agency direc- 
tors can rely for information about EBPs. 

Implementation stage was also associated with posi- 
tion in the overall advice network. The multivariate 
regression model presented in Table 3 reveals that 
county-level and network-position specific variables 
were important independent correlates of implementa- 
tion stage. Individuals in large counties, relative to 
small, reported higher implementation stage, and urban 
counties, relative to rural, reported higher implementa- 
tion stage. Increasing in-degree centrality was positively 
associated with implementation stage at the two-year 
follow-up, while out-degree centrality was not. These 
latter results indicate that, adjusting for county-level 
attributes, being nominated more frequently by others 
in the network was positively associated with implemen- 
tation stage two years later, while the number of nomi- 
nations an individual provided were not associated with 
implementation stage. 



Table 3 Regression of implementation stage on 
centrality, adjusting for county size and urban/rural 
classification (n = 137) 



Variable 


B 


Standard Error 


f value 


p value 


In-degree centrality 


0.16 


0.07 


2.26 


.03 


Out-degree centrality 


0.01 


0.02 


0.61 


.5-1 


Large county 


0.43 


0.14 


3.14 


.00 


Urban county 


0.47 


0.15 


3.24 


.00 



Note: 39 participants are missing from this analysis because their county 
implementation stage could not be identified or they belonged to an 
organization for which implementation stage was not appropriate [e.g., 
California Institute of Mental Health) (F(4) = 13.3, p < .001; R 2 = 0.29). 



Collaboration as critical to evidence-based practice 
implementation 

In addition to identifying the potential predictors of 
implementation stage and supplementing the construc- 
tion of the social networks, the qualitative analysis of 
the semistructured interviews identified features of these 
networks that were critical to the process of EBP imple- 
mentation. Perhaps the most salient of these features 
was the role of collaboration within and between coun- 
ties. Within counties, single agencies often lacked 
resources to implement EBPs independently and noted 
that implementation requires good systems partners. In 
small, rural counties where agencies have limited 
resources to implement EBPs on their own, agency 
directors cited a desire to participate in the Cal-40 
Study in clusters with neighboring counties. 

Poor history of collaboration was often cited as a rea- 
son for failure to implement EBPs. The reasons for the 
lack of collaboration identified by study participants 
included the following: lack of funding to support a colla- 
boration, different priorities and mandates of the colla- 
borating agencies, different organizational cultures of the 
collaborating agencies and the lack of understanding of 
these cultures, and differences in personality and the 
strained relationships caused by these differences. 

Finally, criteria for effective collaborations among agen- 
cies in public youth-serving agencies included individuals 
who can play key roles in the collaborative process, espe- 
cially agency directors and administrators with knowl- 
edge or experience working for another agency who can 
serve as a collaboration broker or facilitator. For example, 
one participant cited her varied experience working for 
multiple agencies as beneficial to understanding complex 
system interactions, stating, "I fortunately have had the 
experience of being a probation officer, a social service 
worker, and a mental health clinician" (Mental Health 
Department Child/ Adolescent Program Chief). 

Role of influence networks in MTFC implementation 

These information and advice networks appear to have 
played an important role in the implementation of 
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MTFC among the first cohort of counties participating 
in the Cal-40 Study. For those who had agreed to parti- 
cipate or were considering participation at the time they 
were interviewed, information about MTFC and the 
Cal-40 Study was obtained from presentations given by 
CIMH representatives at state or regional meetings, 
direct contact by CIMH with county agency directors, 
direct contact by other agency directors within the 
county, or staff within the agency: 

"It came to my attention two different ways. I started 
hearing some discussion about it at the small county asso- 
ciation meetings, which is a break off of the full body 
county Mental Health Directors Association. And I heard 
it from one or two of my peers... Then, the newest program 
manager brought it to my attention. And I think she 
found it on the CIMH website... "-Mental Health Program 
Director 

Only one of the seven systems leaders interviewed from 
the three counties that had either decided not to partici- 
pate in the Cal-40 Study or had not advanced beyond 
stage 1 had received any information about MTFC or the 
Cal-40 Study. 

Discussion 

The results of this study suggest that the structure and 
operation of social networks-specifically, higher in-degree 
centrality of network members, as well as network context, 
reflected in the size of county and whether it was predo- 
minately urban or rural-are central to implementation of 
EBPs. Further, social networks influence the implementa- 
tion process through two mechanisms, development and 
operation of successful collaborations and acquisition of 
information and support related to EBPs. Although many 
factors influence the diffusion of EBPs, researchers have 
consistently found that interpersonal contacts within and 
between organizations and communities are important 
influences on the adoption of new behaviors [7,8,33-36]. 

In this study, the majority of network ties occurred 
within the same county and same implementation stage. 
This is understandable given that both randomization 
and use of the SIC measurement in the Cal-40 Study 
occurred at the county level [19]. However, only a little 
over one-third of network ties existed among individuals 
in the same agency. This could be accounted for, in part, 
by the Cal-40 Study requirements that at least two of the 
three agencies in a county had to agree to participate, 
one of which had to be the mental health agency, in 
order to enroll in the study [19]. The results also sup- 
ported the importance of collaboration between agencies. 
This was reflected in the number of ties among indivi- 
duals representing different agencies in the same county 
and the qualitative data highlighting the importance of 
collaboration for EBP implementation, especially in 
resource-poor rural counties, even when participation of 



more than one agency is not a requirement for imple- 
mentation of a specific EBP. 

The results of this study also help us to understand the 
context in which these networks influence the implemen- 
tation of EBPs and how differences in context, like the size 
of a county or the structure of personal networks, can 
influence whether or not EBPs are adopted by public 
youth-serving agencies. Our results suggest that character- 
istics of the county and in-degree centrality are associated 
with EBP implementation stage. Characteristics of the 
county include its size and urban/rural status. In our 
sample, larger, urban counties were classified in a higher 
implementation stage than their smaller, rural counter- 
parts. A similar association between county size and days 
to consent to participate in the Cal-40 Study in all three 
California cohorts was reported by Wang and colleagues 
[20]. Analysis of qualitative interviews with systems leaders 
found that small, rural counties often lack the resources to 
implement innovative practices on their own due to a lim- 
ited supply of qualified staff, funding, and available clients. 
The two counties that declined to participate in the Cal-40 
Study were small, rural counties possessing networks that 
were also small and lacking ties to other networks that 
had decided to participate in the study and were proceed- 
ing with MTFC implementation. These findings highlight 
the importance of networks involving ties to counties with 
resources or the pooling of resources via existing net- 
works. These networks also exposed agency directors and 
senior administrators to information about and opportu- 
nities to implement EBPs, which, in turn, influenced deci- 
sions about whether or not to implement these practices. 

However, we also found that MTFC implementation 
stage at the two-year follow-up was associated with posi- 
tion in the overall advice networks at baseline. Higher- 
status individuals, measured by in-degree centrality, were 
more likely to work in counties that achieved a higher 
stage of implementation two years later. These individuals 
were nominated by others as a source of information and 
advice about EBPs and innovative programs in general. 
The central position of these individuals in influence net- 
works makes sense since systems leaders would be 
inclined to seek information and advice from someone 
who had experience and was successful in implementing 
such practices. These findings are also consistent with 
Valente and colleagues' findings of the association between 
the presence of opinion leaders in one's social networks 
and rates of adoption of innovative practices [12-16]. 

Not all opinion leaders need have a high degree of cen- 
trality; in some cases, opinion leaders are persons who 
bridge different social networks, and their position as a 
bridging tie facilitates their success in bringing new prac- 
tices from one network to another [37]. There are several 
nodes in this network whose structural position could 
allow for such bridging between sub-networks. Further, 
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although our results point to an association between 
stage of implementation and in- degree centrality but not 
out-degree centrality, it is possible that these two forms 
of status operate differently at different stages of imple- 
mentation, with the former being more important in the 
earlier stages and the latter being more important in sub- 
sequent stages. 

Our study results also provide an indication of how 
influence networks operate to implement EBPs. The semi- 
structured interviews provided numerous instances of 
exchange of information within agencies, within counties, 
and across counties. This exchange usually occurred 
through regularly scheduled meetings or conferences, 
through a search for information concerning the EBP by 
the systems leader, or through dissemination efforts of 
intermediary organizations like CIMH. Influence networks 
also operate to implement EBPs by sharing resources, 
which include funding, staffing, or consumers. This shar- 
ing is easier in large counties because agencies in these 
counties possess more resources than similar agencies in 
small counties. However, the existence of subgroups or cli- 
ques may preclude sharing due to competition for the 
same resources. In smaller, rural counties, on the other 
hand, resources are often shared between agencies in the 
same counties or with agencies in neighboring counties 
because the individual agency frequently lacks the capital, 
staffing, or consumer demand necessary to initiate or sus- 
tain implementation efforts. 

Implementation was also associated with greater con- 
nectivity across counties. Counties who declined to partici- 
pate or did not advance beyond stage 1 had no ties or links 
outside the county. In contrast, counties that had achieved 
stage 6 or higher were all linked to CIMH, a primary 
source of information on EBPs in the state. Most of the 
network links to CIMH were with county mental health 
agency leaders, which is understandable given the involve- 
ment of CIMH in regularly scheduled meetings of the 
California Mental Health Directors Association and with 
county Chief Probation Officers, which can also be 
explained by the fact that the key CIMH "node" was a for- 
mer county chief probation officer. 

One of the conclusions to be drawn from this research 
is that implementation strategies should be designed to 
either build influence networks or capitalize on existing 
networks. The CDT approach being tested in the parent 
study is designed to build social networks that offer sup- 
port to network members in implementing EBPs. Other 
strategies with a similar aim include the Institute for 
Healthcare Innovation's Breakthrough Series collabora- 
tive [38]. Dissemination efforts can and should make use 
of existing networks. For instance, as revealed in the 
interviews with systems leaders in this study, networks 
provide access to opportunities to observe firsthand the 
implementation and effectiveness of EBPs in systems that 



are regarded as models or early adopters. Strategies for 
implementation should strive to create partnerships 
between agencies within counties that serve the same tar- 
get population and build influence networks across coun- 
ties, thereby enabling systems leaders in agencies based in 
small rural counties or possessing small influence net- 
works to acquire more information and resources from 
leaders in agencies based in large urban counties. 

There are several limitations to our study that deserve 
mention. First, this investigation was conducted during 
the initial or first steps of EBP implementation with a 
small number of counties. Although our findings suggest 
that there will be changes in patterns and processes of 
implementation over time, we were primarily interested 
in examining networks at the initial stages of the imple- 
mentation process and then determining whether these 
"baseline" networks could predict the implementation 
trajectory over a two-year period. Second, systems leaders 
who participated in interviews at this stage of the Cal-40 
Study represent almost all of the first cohort but may not 
represent the broader population of systems leaders par- 
ticipating in other cohorts, much less the broader popu- 
lation of systems leaders engaged in child and adolescent 
mental health services. Thus, the results obtained thus 
far may not generalize to either population, although 
cohort 1 counties were selected through a process of ran- 
domization and thus should be representative of all 40 
counties participating in the parent study. Third, the 
176-member network was constructed based on informa- 
tion from 38 interviewees who were not asked to provide 
information on sociodemographic and occupational char- 
acteristics on those they nominated. Consequently, we 
lacked individual-level measures on some of the nodes 
who were not directly interviewed, thereby limiting our 
statistical power to examine the influence of such charac- 
teristics as predictors of network structure or implemen- 
tation stage. Finally, both collection and interpretation of 
qualitative data is susceptible to subjective bias and pre- 
conceived ideas of the investigators. However, the use of 
multiple observers as well as multiple sources of data to 
achieve "triangulation" [39] should minimize such bias. 

Conclusions 

Despite these limitations, the results of this study suggest 
that social networking is central to implementation of 
EBPs through two mechanisms: development and opera- 
tion of successful collaborations and acquisition of infor- 
mation and support related to EBPs. The most influential 
networks appear to be those that extend beyond service- 
system jurisdictions. This study helps us to understand the 
context in which these networks influence EBP implemen- 
tation and how differences in context of personal networks 
can influence whether or not EBPs are adopted by public 
youth-serving agencies. It also helps to inform the design 
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of implementation strategies that either build influence 
networks or capitalize on existing networks. 
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